Query generation for collaborative datasets

ABSTRACT

Various embodiments relate generally to data science and data analysis, computer software and systems, and wired and wireless network communications to provide an interface between repositories of disparate datasets and computing machine-based entities that seek access to the datasets, and, more specifically, to a computing and data storage platform that facilitates consolidation of one or more datasets, whereby a collaborative data layer and associated logic facilitate, for example, efficient access to, and implementation of, collaborative datasets. In some examples, a method may include receiving data representing a query of a consolidated dataset that may include datasets formatted atomized datasets, analyzing the query to classify portions of the query to form classified query portions, partitioning the query into sub-queries as a function of a classification type for each of the classified query portions, and retrieving data representing a query result from distributed data repositories.

CROSS-REFERENCE TO APPLICATIONS

This application is a continuation application of U.S. patentapplication Ser. No. 15/186,517 filed on Jun. 19, 2016, and titled“QUERY GENERATION FOR COLLABORATIVE DATASETS,” which is hereinincorporated by reference in its entirety for all purposes.

FIELD

Various embodiments relate generally to data science and data analysis,computer software and systems, and wired and wireless networkcommunications to provide an interface between repositories of disparatedatasets and computing machine-based entities that seek access to thedatasets, and, more specifically, to a computing and data storageplatform that facilitates consolidation of one or more datasets, wherebya collaborative data layer and associated logic facilitate, for example,efficient access to collaborative datasets.

BACKGROUND

Advances in computing hardware and software have fueled exponentialgrowth in the generation of vast amounts of data due to increasedcomputations and analyses in numerous areas, such as in the variousscientific and engineering disciplines, as well as in the application ofdata science techniques to endeavors of good-will (e.g., areas ofhumanitarian, environmental, medical, social, etc.). Also, advances inconventional data storage technologies provide the ability to store theincreasing amounts of generated data. Consequently, traditional datastorage and computing technologies have given rise to a phenomenonnumerous desperate datasets that have reached sizes (e.g., includingtrillions of gigabytes of data) and complexity that traditiondata-accessing and analytic techniques are generally not well-suited forassessing conventional datasets.

Conventional technologies for implementing datasets typically rely ondifferent computing platforms and systems, different databasetechnologies, and different data formats, such as CSV, HTML, JSON, XML,etc. Further, known data-distributing technologies are not well-suitedto enable interoperability among datasets. Thus, many typical datasetsare warehouses or otherwise reside in conventional data stores as “datasilos,” which describe insulated data systems and datasets that aregenerally incompatible or inadequate to facilitate datainteroperability. Moreover, corporate-generated datasets generally mayreside in data silos to preserve commercial advantages, even though thesharing of some of the corporate-generated datasets may provide littleto no commercial disadvantage and otherwise might provide publicbenefits if shared altruistically. Additionally, academia-generateddatasets also may generally reside in data silos due to limitedcomputing and data system resources and to preserve confidentialityprior to publications of, for example, journals and other academicresearch papers. While researchers may make their data for availableafter publication, the form of the data and datasets are not well-suitedfor access and implementation with other sources of data.

Conventional approaches to provide dataset generation and management,while functional, suffer a number of other drawbacks. For example,individuals or organizations, such as non-profit organizations, usuallyhave limited resources and skills to operate the traditional computingand data systems, thereby hindering their access to information thatmight otherwise provide tremendous benefits. Also, creators of datasetstend to do so for limited purposes, and once the dataset is created,knowledge related to the sources of data and the manner of constructingthe dataset is lost. In other examples, some conventional approachesprovide remote data storage (e.g., “cloud”-based data storage) tocollect differently-formatted repositories of data, however, theseapproaches are not well-suited to resolve sufficiently the drawbacks oftraditional techniques of dataset generation and management.

Thus, what is needed is a solution for facilitating techniques togenerate, locate, and access datasets, without the limitations ofconventional techniques.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments or examples (“examples”) of the invention aredisclosed in the following detailed description and the accompanyingdrawings:

FIG. 1 is a diagram depicting a collaborative dataset consolidationsystem, according to some embodiments;

FIG. 2 is a diagram depicting an example of an atomized data point,according to some embodiments;

FIG. 3 is a diagram depicting an example of a flow chart to perform aquery operation against collaborative datasets, according to someembodiments;

FIG. 4 is a diagram depicting operation an example of a collaborativedataset consolidation system, according to some examples;

FIG. 5 is a diagram depicting a flow chart to perform an operation of acollaborative dataset consolidation system, according to someembodiments;

FIG. 6 is a diagram depicting an example of a dataset analyzer and aninference engine, according to some embodiments;

FIG. 7 is a diagram depicting operation of an example of an inferenceengine, according to some embodiments;

FIG. 8 is a diagram depicting a flow chart as an example of ingesting anenhanced dataset into a collaborative dataset consolidation system,according to some embodiments;

FIG. 9 is a diagram depicting an example of a dataset ingestioncontroller, according to various embodiments;

FIG. 10 is a diagram depicting a flow chart as an example of managingversioning of dataset, according to some embodiments;

FIG. 11 is a diagram depicting an example of an atomized data-basedworkflow loader, according to various embodiments;

FIG. 12 is a diagram depicting a flow chart as an example of loading anatomized dataset into an atomized data point store, according to someembodiments;

FIG. 13 is a diagram depicting an example of a dataset query engine,according to some embodiments;

FIG. 14 is a diagram depicting a flow chart as an example of querying anatomized dataset stored in an atomized data point store, according tosome embodiments;

FIG. 15 is a diagram depicting an example of a collaboration managerconfigured to present collaborative information regarding collaborativedatasets, according to some embodiments; and

FIG. 16 illustrates examples of various computing platforms configuredto provide various functionalities to components of a collaborativedataset consolidation system, according to various embodiments.

DETAILED DESCRIPTION

Various embodiments or examples may be implemented in numerous ways,including as a system, a process, an apparatus, a user interface, or aseries of program instructions on a computer readable medium such as acomputer readable storage medium or a computer network where the programinstructions are sent over optical, electronic, or wirelesscommunication links. In general, operations of disclosed processes maybe performed in an arbitrary order, unless otherwise provided in theclaims.

A detailed description of one or more examples is provided below alongwith accompanying figures. The detailed description is provided inconnection with such examples, but is not limited to any particularexample. The scope is limited only by the claims, and numerousalternatives, modifications, and equivalents thereof. Numerous specificdetails are set forth in the following description in order to provide athorough understanding. These details are provided for the purpose ofexample and the described techniques may be practiced according to theclaims without some or all of these specific details. For clarity,technical material that is known in the technical fields related to theexamples has not been described in detail to avoid unnecessarilyobscuring the description.

FIG. 1 is a diagram depicting a collaborative dataset consolidationsystem, according to some embodiments. Diagram 100 depicts an example ofcollaborative dataset consolidation system 110 that may be configured toconsolidate one or more datasets to form collaborative datasets. Acollaborative dataset, according to some non-limiting examples, is a setof data that may be configured to facilitate data interoperability overdisparate computing system platforms, architectures, and data storagedevices. Further, a collaborative dataset may also be associated withdata configured to establish one or more associations (e.g., metadata)among subsets of dataset attribute data for datasets, whereby attributedata may be used to determine correlations (e.g., data patterns, trends,etc.) among the collaborative datasets. Collaborative datasetconsolidation system 110 may then present the correlations via computingdevices 109 a and 109 b to disseminate dataset-related information toone or more users 108 a and 108 b. Thus, a community of users 108, aswell as any other participating user, may discover and sharedataset-related information of interest in association withcollaborative datasets. Collaborative datasets, with or withoutassociated dataset attribute data, may be used to facilitate easiercollaborative dataset interoperability among sources of data that may bedifferently formatted at origination. According to various embodiments,one or more structural and/or functional elements described in FIG. 1,as well as below, may be implemented in hardware or software, or both.

Collaborative dataset consolidation system 110 is depicted as includinga dataset ingestion controller 120, a dataset query engine 130, acollaboration manager 160, a collaborative data repository 162, and adata repository 140, according to the example shown. Dataset ingestioncontroller 120 may be configured to receive data representing a dataset104 a having, for example, a particular data format (e.g., CSV, XML,JSON, XLS, MySQL, binary, etc.), and may be further configured toconvert dataset 104 a into a collaborative data format for storage in aportion of data arrangement 142 a in repository 140. According to someembodiments, a collaborative data format may be configured to, but neednot be required to, format converted dataset 104 a as an atomizeddataset. An atomized dataset may include a data arrangement in whichdata is stored as an atomized data point 114 that, for example, may bean irreducible or simplest representation of data that may be linkableto other atomized data points, according to some embodiments. Atomizeddata point 114 may be implemented as a triple or any other datarelationship that expresses or implements, for example, a smallestirreducible representation for a binary relationship between two dataunits. As atomized data points may be linked to each other, dataarrangement 142 a may be represented as a graph, whereby the converteddataset 104 a (i.e., atomized dataset 104 a) forms a portion of thegraph. In some cases, an atomized dataset facilitates merging of datairrespective of whether, for example, schemas or applications differ.

Further, dataset ingestion controller 120 may be configured to identifyother datasets that may be relevant to dataset 104 a. In oneimplementation, dataset ingestion controller 120 may be configured toidentify associations, links, references, pointers, etc. that mayindicate, for example, similar subject matter between dataset 104 a anda subset of other datasets (e.g., within or without repository 140). Insome examples, dataset ingestion controller 120 may be configured tocorrelate dataset attributes of an atomized data set with other atomizeddatasets or non-atomized datasets. Dataset ingestion controller 120 orother any other component of collaborative dataset consolidation system110 may be configured to format or convert a non-atomized dataset (orany other differently-formatted dataset) into a format similar to thatof converted dataset 104 a). Therefore, dataset ingestion controller 120may determine or otherwise use associations to consolidate datasets toform, for example, consolidated datasets 132 a and consolidated datasets132 b.

As shown in diagram 100, dataset ingestion controller 120 may beconfigured to extend a dataset (i.e., the converted dataset 104 a storedin data arrangement 142 a) to include, reference, combine, orconsolidate with other datasets within data arrangement 142 a orexternal thereto. Specifically, dataset ingestion controller 120 mayextend an atomized dataset 104 a to form a larger or enriched dataset,by associating or linking (e.g., via links 111) to other datasets, suchas external entity datasets 104 b, 104 c, and 104 n, form one or moreconsolidated datasets. Note that external entity datasets 104 b, 104 c,and 104 n may be converted to form external datasets atomized datasets142 b, 142 c, and 142 n, respectively. The term “external dataset,” atleast in this case, can refer to a dataset generated externally tosystem 110 and may or may not be formatted as an atomized dataset.

As shown, different entities 105 a, 105 b, and 105 n each include acomputing device 102 (e.g., representative of one or more servers and/ordata processors) and one or more data storage devices 103 (e.g.,representative of one or more database and/or data store technologies).Examples of entities 105 a, 105 b, and 105 n include individuals, suchas data scientists and statisticians, corporations, universities,governments, etc. A user 101 a, 101 b, and 101 n (and associated useraccount identifiers) may interact with entities 105 a, 105 b, and 105 n,respectively. Each of entities 105 a, 105 b, and 105 n may be configuredto perform one or more of the following: generating datasets, modifyingdatasets, querying datasets, analyzing datasets, hosting datasets, andthe like, whereby one or more entity datasets 104 b, 104 c, and 104 nmay be formatted in different data formats. In some cases, these formatsmay be incompatible for implementation with data stored in repository140. As shown, differently-formatted datasets 104 b, 104 c, and 104 nmay be converted into atomized datasets, each of which is depicted indiagram 100 as being disposed in a dataspace. Namely, atomized datasets142 b, 142 c, and 142 n are depicted as residing in dataspaces 113 a,113 b, and 113 n, respectively. In some examples, atomized datasets 142b, 142 c, and 142 n may be represented as graphs.

According to some embodiments, atomized datasets 142 b, 142 c, and 142 nmay be imported into collaborative dataset consolidation system 110 forstorage in one or more repositories 140. In this case, dataset ingestioncontroller 120 may be configured to receive entity datasets 104 b, 104c, and 104 n for conversion into atomized datasets, as depicted incorresponding dataspaces 113 a, 113 b, and 113 n. Collaborative dataconsolidation system 110 may store atomized datasets 142 b, 142 c, and142 n in repository 140 (i.e., internal to system 110) or may providethe atomized datasets for storage in respective entities 105 a, 105 b,and 105 n (i.e., without or external to system 110). Alternatively, anyof entities 105 a, 105 b, and 105 n may be configured to convert entitydatasets 104 b, 104 c, and 104 n and store corresponding atomizeddatasets 142 b, 142 c, and 142 n in one or more data storage devices103. In this case, atomized datasets 142 b, 142 c, and 142 n may behosted for access by dataset ingestion controller 120 for linking vialinks 111 to extend datasets with data arrangement 142 a.

Thus, collaborative dataset consolidation system 110 is configured toconsolidate datasets from a variety of different sources and in avariety of different data formats to form consolidated datasets 132 aand 132 b. As shown, consolidated dataset 132 a extends a portion ofdataset in data arrangement 142 a to include portions of atomizeddatasets 142 b, 142 c, and 142 n via links 111, whereas consolidateddataset 132 b extends another portion of a dataset in data arrangement142 a to include other portions of atomized datasets 142 b and 142 c vialinks 111. Note that entity dataset 104 n includes a secured set ofprotected data 131 c that may require a level of authorization orauthentication to access. Without authorization, link 119 cannot beimplemented to access protected data 131 c. For example, user 101 n maybe a system administrator that may program computing device 102 n torequire authorization to gain access to protected data 131 c. In somecases, dataset ingestion controller 120 may or may not provide anindication that link 119 exists based on whether, for example, user 108a has authorization to form a consolidated dataset 132 b to includeprotected data 131 c.

Dataset query engine 130 may be configured to generate one or morequeries, responsive to receiving data representing one or more queriesvia computing device 109 a from user 108 a. Dataset query engine 130 isconfigured to apply query data to one or more collaborative datasets,such as consolidated dataset 132 a and consolidated dataset 132 b, toaccess the data therein to generate query response data 112, which maybe presented via computing device 109 a to user 108 a. According to someexamples, dataset query engine 130 may be configured to identify one ormore collaborative datasets subject to a query to either facilitate anoptimized query or determine authorization to access one or more of thedatasets, or both. As to the latter, dataset query engine 130 may beconfigured to determine whether one of users 108 a and 108 b isauthorized to include protected data 131 c in a query of consolidateddataset 132 b, whereby the determination may be made at the time (orsubstantially at the time) dataset query engine 130 identifies one ormore datasets subject to a query.

Collaboration manager 160 may be configured to assign or identify one ormore attributes associated with a dataset, such as a collaborativedataset, and may be further configured to store dataset attributes ascollaborative data in repository 162. Examples of dataset attributesinclude, but are not limited to, data representing a user accountidentifier, a user identity (and associated user attributes, such as auser first name, a user last name, a user residential address, aphysical or physiological characteristics of a user, etc.), one or moreother datasets linked to a particular dataset, one or more other useraccount identifiers that may be associated with the one or moredatasets, data-related activities associated with a dataset (e.g.,identity of a user account identifier associated with creating,modifying, querying, etc. a particular dataset), and other similarattributes. Another example of a dataset attribute is a “usage” or typeof usage associated with a dataset. For instance, a virus-relateddataset (e.g., Zika dataset) may have an attribute describing usage tounderstand victim characteristics (i.e., to determine a level ofsusceptibility), an attribute describing usage to identify a vaccine, anattribute describing usage to determine an evolutionary history ororigination of the Zika, SARS, MERS, HIV, or other viruses, etc.Further, collaboration manager 160 may be configured to monitor updatesto dataset attributes to disseminate the updates to a community ofnetworked users or participants. Therefore, users 108 a and 108 b, aswell as any other user or authorized participant, may receivecommunications (e.g., via user interface) to discover new orrecently-modified dataset-related information in real-time (or nearreal-time).

In view of the foregoing, the structures and/or functionalities depictedin FIG. 1 illustrate a dataset consolidated system that may beconfigured to consolidate datasets originating in different data formatswith different data technologies, whereby the datasets (e.g., ascollaborative datasets) may originate external to the system.Collaborative dataset consolidation system 110, therefore, may beconfigured to extend a dataset beyond its initial quantity and quality(e.g., types of data, etc.) of data to include data from other datasets(e.g., atomized datasets) linked to the dataset to form a consolidateddataset. Note that while a consolidated dataset may be configured topersist in repository 140 as a contiguous dataset, collaborative datasetconsolidation system 110 is configured to store at least one of atomizeddatasets 142 a, 142 b, 142 c, and 142 n (e.g., one or more of atomizeddatasets 142 a, 142 b, 142 c, and 142 n may be stored internally orexternally) as well data representing links 111. Hence, at a given pointin time (e.g., during a query), the data associated one of atomizeddatasets 142 a, 142 b, 142 c, and 142 n may be loaded into an atomicdata store against which the query can be performed. Therefore,collaborative dataset consolidation system 110 need not be required togenerate massive graphs based on numerous datasets, but rather,collaborative dataset consolidation system 110 may create a graph basedon a consolidated dataset in one operational state (of a number ofoperational states), and can be partitioned in another operational state(but can be linked via links 111 to form the graph). In some cases,different graph portions may persist separately and may be linkedtogether when loaded into a data store to provide resources for a query.Further, collaborative dataset consolidation system 110 may beconfigured to extend a dataset beyond its initial quantity and qualityof data based on using atomized datasets that include atomized datapoints (e.g., as an addressable data unit or fact), which facilitateslinking, joining, or merging the data from disparate data formats ordata technologies (e.g., different schemas or applications for which adataset is formatted). Atomized datasets facilitate datainteroperability over disparate computing system platforms,architectures, and data storage devices, according to variousembodiments.

According to some embodiments, collaborative dataset consolidationsystem 110 may be configured to provide a granular level of securitywith which an access to each dataset is determined on adataset-by-dataset basis (e.g., per-user access or per-user accountidentifier to establish per-dataset authorization). Therefore, a usermay be required to have per-dataset authorization to access a group ofdatasets less than a total number of datasets (including a singledataset). In some examples, dataset query engine 130 may be configuredto assert query-level authorization or authentication. As such,non-users (e.g., participants) without account identifiers (or userswithout authentication) may apply a query (e.g., limited to a query, forexample) to repository 140 without receiving authorization to accesssystem 110 generally. Dataset query engine 130 may implement such aquery so long as the query includes, or is otherwise associated with,authorization data.

Collaboration manager 160 may be configured as, or to implement, acollaborative data layer and associated logic to implement collaborativedatasets for facilitating collaboration among consumers of datasets. Forexample, collaboration manager 160 may be configured to establish one ormore associations (e.g., as metadata) among dataset attribute data (fora dataset) and/or other attribute data (for other datasets (e.g., withinor without system 110)). As such, collaboration manager 160 candetermine a correlation between data of one dataset to a subset of otherdatasets. In some cases, collaboration manager 160 may identify andpromote a newly-discovered correlation to users associated with a subsetof other databases. Or, collaboration manager 160 may disseminateinformation about activities (e.g., name of a user performing a query,types of data operations performed on a dataset, modifications to adataset, etc.) for a particular dataset. To illustrate, consider thatuser 108 a is situated in South America and is querying arecently-generated dataset that includes data about the Zika virus overdifferent age ranges and genders over various population ranges.Further, consider that user 108 b is situated in North America and alsohas generated or curated datasets directed to the Zika virus.Collaborative dataset consolidation system 110 may be configured todetermine a correlation between the datasets of users 108 a and 108 b(i.e., subsets of data may be classified or annotated as Zika-related).System 110 also may optionally determine whether user 108 b hasinteracted with the newly-generated dataset about the Zika virus(whether user, for example, viewed, queried, searched, etc. thedataset). Regardless, collaboration manager 160 may generate anotification to present in a user interface 118 of computing device 109b. As shown, user 108 b is informed in an “activity feed” portion 116 ofuser interface 118 that “Dataset X” has been queried and is recommendedto user 108 b (e.g., based on the correlated scientific and researchinterests related to the Zika virus). User 108 b, in turn, may modifyDataset X to form Dataset XX, thereby enabling a community ofresearchers to expeditiously access datasets (e.g., previously-unknownor newly-formed datasets) as they are generated to facilitate scientificcollaborations, such as developing a vaccine for the Zika virus. Notethat users 101 a, 101 b, and 101 n may also receive similarnotifications or information, at least some of which present one or moreopportunities to collaborate and use, modify, and share datasets in a“viral” fashion. Therefore, collaboration manager 160 and/or otherportions of collaborative dataset consolidation system 110 may providecollaborative data and logic layers to implement a “social network” fordatasets.

FIG. 2 is a diagram depicting an example of an atomized data point,according to some embodiments. Diagram 200 depicts a portion 201 of anatomized dataset that includes an atomized data point 214. In someexamples, the atomized dataset is formed by converting a data formatinto a format associated with the atomized dataset. In some cases,portion 201 of the atomized dataset can describe a portion of a graphthat includes one or more subsets of linked data. Further to diagram200, one example of atomized data point 214 is shown as a datarepresentation 214 a, which may be represented by data representing twodata units 202 a and 202 b (e.g., objects) that may be associated viadata representing an association 204 with each other. One or moreelements of data representation 214 a may be configured to beindividually and uniquely identifiable (e.g., addressable), eitherlocally or globally in a namespace of any size. For example, elements ofdata representation 214 a may be identified by identifier data 290 a,290 b, and 290 c.

In some embodiments, atomized data point 214 a may be associated withancillary data 203 to implement one or more ancillary data functions.For example, consider that association 204 spans over a boundary betweenan internal dataset, which may include data unit 202 a, and an externaldataset (e.g., external to a collaboration dataset consolidation), whichmay include data unit 202 b. Ancillary data 203 may interrelate viarelationship 280 with one or more elements of atomized data point 214 asuch that when data operations regarding atomized data point 214 a areimplemented, ancillary data 203 may be contemporaneously (orsubstantially contemporaneously) accessed to influence or control a dataoperation. In one example, a data operation may be a query and ancillarydata 203 may include data representing authorization (e.g., credentialdata) to access atomized data point 214 a at a query-level dataoperation (e.g., at a query proxy during a query). Thus, atomized datapoint 214 a can be accessed if credential data related to ancillary data203 is valid (otherwise, a query with which authorization data is absentmay be rejected or invalidated). According to some embodiments,credential data, which may or may not be encrypted, may be integratedinto or otherwise embedded in one or more of identifier data 290 a, 290b, and 290 c. Ancillary data 203 may be disposed in other data portionof atomized data point 214 a, or may be linked (e.g., via a pointer) toa data vault that may contain data representing access permissions orcredentials.

Atomized data point 214 a may be implemented in accordance with (or becompatible with) a Resource Description Framework (“RDF”) data model andspecification, according to some embodiments. An example of an RDF datamodel and specification is maintained by the World Wide Web Consortium(“W3C”), which is an international standards community of Memberorganizations. In some examples, atomized data point 214 a may beexpressed in accordance with Turtle, RDF/XML, N-Triples, N3, or otherlike RDF-related formats. As such, data unit 202 a, association 204, anddata unit 202 b may be referred to as a “subject,” “predicate,” and“object,” respectively, in a “triple” data point. In some examples, oneor more of identifier data 290 a, 290 b, and 290 c may be implementedas, for example, a Uniform Resource Identifier (“URI”), thespecification of which is maintained by the Internet Engineering TaskForce (“IETF”). According to some examples, credential information(e.g., ancillary data 203) may be embedded in a link or a URI (or in aURL) for purposes of authorizing data access and other data processes.Therefore, an atomized data point 214 may be equivalent to a triple datapoint of the Resource Description Framework (“RDF”) data model andspecification, according to some examples. Note that the term “atomized”may be used to describe a data point or a dataset composed of datapoints represented by a relatively small unit of data. As such, an“atomized” data point is not intended to be limited to a “triple” or tobe compliant with RDF; further, an “atomized” dataset is not intended tobe limited to RDF-based datasets or their variants. Also, an “atomized”data store is not intended to be limited to a “triplestore,” but theseterms are intended to be broader to encompass other equivalent datarepresentations.

FIG. 3 is a diagram depicting an example of a flow chart to perform aquery operation against collaborative datasets, according to someembodiments. Diagram 300 depicts a flow for an example of queryingcollaborative datasets in association with a collaborative datasetconsolidation system. At 302, data representing a query may be receivedinto a collaborative dataset consolidation system to query an atomizeddataset. The atomized dataset may include subsets of linked atomizeddata points. In some examples, the dataset may be associated with orcorrelated to an identifier, such as a user account identifier or adataset identifier. At 304, datasets relevant to the query areidentified. The datasets may be disposed in disparate data repositories,regardless of whether internal to a system or external thereto. In somecases, a dataset relevant to a query may be identified by the useraccount identifier, the dataset identifier, or any other data (e.g.,metadata or attribute data) that may describe data types and dataclassifications of the data in the datasets.

In some cases, at 304, a subset of data attributes associated with thequery may be determined, such as a description or annotation of the datathe subset of data attributes. To illustrate, consider an example inwhich the subset of data attributes includes data types orclassifications that may be found as column in a tabular data format(e.g., prior to atomization or as an alternate view). The collaborativedataset consolidation system may then retrieve a subset of atomizeddatasets that include data equivalent to (or associated with) one ormore of the data attributes. So if the subset of data attributesincludes alphanumeric characters (e.g., two-letter codes, such as “AF”for Afghanistan), then the column can be identified as including countrycode data. Based on the country codes as a “data classification,” thecollaborative dataset consolidation system may correlate country codedata in other atomized datasets to the dataset (e.g., the querieddataset). Then, the system may retrieve additional atomized datasetsthat include country codes to form a consolidated dataset. Thus, thesedatasets may be linked together by country codes. Note that in somecases, the system may implement logic to “infer” that two letters in a“column of data” of a tabular, pre-atomized dataset includes countrycodes. As such, the system may “derive” an annotation (e.g., a data typeor classification) as a “country code.” A dataset ingestion controllermay be configured to analyze data and/or data attributes to correlatethe same over multiple datasets, the dataset ingestion controller beingfurther configured to infer a data type or classification of a groupingof data (e.g., data disposed in a column or any other data arrangement),according to some embodiments.

At 306, a level of authorization associated with the identifier may beidentified to facilitate access to one or more of the datasets for thequery. At, 308, one or more queries may be generated based on a querythat may be configured to access the disparate data repositories. Atleast one of the one or more queries may be formed (e.g., rewritten) asa sub-query. That is, a sub-query may be generated to access aparticular data type stored in a particular database engine or datastore, either of which may be architected to accommodate a particulardata type (e.g., data relating to time-series data, GPU-relatedprocessing data, geo-spatial-related data, etc.). At 310, datarepresenting query results from the disparate data repositories may beretrieved. Note that a data repository from which a portion of the queryresults are retrieved may be disposed external to a collaborativedataset consolidation system. Further, data representing attributes orcharacteristics of the query may be passed to collaboration manager,which, in turn, may inform interested users of activities related to thedataset.

FIG. 4 is a diagram depicting operation an example of a collaborativedataset consolidation system, according to some examples. Diagram 400includes a collaborative dataset consolidation system 410, which, inturn, includes a dataset ingestion controller 420, a collaborationmanager 460, a dataset query engine 430, and a repository 440, which mayrepresent one or more data stores. In the example shown, consider that auser 408 b, which is associated with a user account data 407, may beauthorized to access (via networked computing device 409 b)collaborative dataset consolidation system to create a dataset and toperform a query. User interface 418 a of computing device 409 b mayreceive a user input signal to activate the ingestion of a data file,such as a CSV formatted file (e.g., “XXX.csv”). Hence, dataset ingestioncontroller 420 may receive data 401 a representing the CSV file and mayanalyze the data to determine dataset attributes. Examples of datasetattributes include annotations, data classifications, data types, anumber of data points, a number of columns, a “shape” or distribution ofdata and/or data values, a normative rating (e.g., a number between 1 to10 (e.g., as provided by other users)) indicative of the “applicability”or “quality” of the dataset, a number of queries associated with adataset, a number of dataset versions, identities of users (orassociated user identifiers) that analyzed a dataset, a number of usercomments related to a dataset, etc.). Dataset ingestion controller 420may also convert the format of data file 401 a to an atomized dataformat to form data representing an atomized dataset 401 b that may bestored as dataset 442 a in repository 440.

As part of its processing, dataset ingestion controller 420 maydetermine that an unspecified column of data 401 a, which includes five(5) integer digits, is a column of “zip code” data. As such, datasetingestion controller 420 may be configured to derive a dataclassification or data type “zip code” with which each set of 5 digitscan be annotated or associated. Further to the example, consider thatdataset ingestion controller 420 may determine that, for example, basedon dataset attributes associated with data 401 a (e.g., zip code as anattribute), both a public dataset 442 b in external repositories 440 aand a private dataset 442 c in external repositories 440 b may bedetermined to be relevant to data file 401 a. Individuals 408 c, via anetworked computing system, may own, maintain, administer, host orperform other activities in association with public dataset 442 b.Individual 408 d, via a networked computing system, may also own,maintain, administer, and/or host private dataset 442 c, as well asrestrict access through a secured boundary 415 to permit authorizedusage.

Continuing with the example, public dataset 442 b and private dataset442 c may include “zip code”-related data (i.e., data identified orannotated as zip codes). Dataset ingestion controller 420 generates adata message 402 a that includes an indication that public dataset 442 band/or private dataset 442 c may be relevant to the pending uploadeddata file 401 a (e.g., datasets 442 b and 442 c include zip codes).Collaboration manager 460 receive data message 402 a, and, in turn, maygenerate user interface-related data 403 a to cause presentation of anotification and user input data configured to accept user input at userinterface 418 b.

If user 408 b wishes to “enrich” dataset 401 a, user 408 b may activatea user input (not shown on interface 418 b) to generate a user inputsignal data 403 b indicating a request to link to one or more otherdatasets. Collaboration manager 460 may receive user input signal data403 b, and, in turn, may generate instruction data 402 b to generate anassociation (or link 441 a) between atomized dataset 442 a and publicdataset 442 b to form a consolidated dataset, thereby extending thedataset of user 408 b to include knowledge embodied in externalrepositories 440 a. Therefore, user 408 b's dataset may be generated asa collaborative dataset as it may be based on the collaboration withpublic dataset 442 b, and, to some degree, its creators, individuals 408c. Note that while public dataset 442 b may be shown external to system410, public dataset 442 b may be ingested via dataset ingestioncontroller 420 for storage as another atomized dataset in repository440. Or, public dataset 442 b may be imported into system 410 as anatomized dataset in repository 440 (e.g., link 411 a is disposed withinsystem 410). Similarly, if user 408 b wishes to “enrich” atomizeddataset 401 b with private dataset 442 c, user 408 b may extend itsdataset 442 a by forming a link 411 b to private dataset 442 c to form acollaborative dataset. In particular, dataset 442 a and private dataset442 c may consolidate to form a collaborative dataset (e.g., dataset 442a and private dataset 442 c are linked to facilitate collaborationbetween users 408 b and 408 d). Note that access to private dataset 442c may require credential data 417 to permit authorization to passthrough secured boundary 415. Note, too, that while private dataset 442c may be shown external to system 410, private dataset 442 c may beingested via dataset ingestion controller 420 for storage as anotheratomized dataset in repository 440. Or, private dataset 442 c may beimported into system 410 as an atomized dataset in repository 440 (e.g.,link 411 b is disposed within system 410). According to some examples,credential data 417 may be required even if private dataset 442 c isstored in repository 440. Therefore, user 408 d may maintain dominion(e.g., ownership and control of access rights or privileges, etc.) of anatomized version of private dataset 442 c when stored in repository 440.

Should user 408 b desire not to link dataset 442 a with other datasets,then upon receiving user input signal data 403 b indicating the same,dataset ingestion controller 420 may store dataset 401 b as atomizeddataset 442 a without links (or without active links) to public dataset442 b or private dataset 442 c. Thereafter, user 408 b may issue viacomputing device 409 b query data 404 a to dataset query engine 430,which may be configured to apply one or more queries to dataset 442 a toreceive query results 404 b. Note that dataset ingestion controller 420need not be limited to performing the above-described function duringcreation of a dataset. Rather, dataset ingestion controller 420 maycontinually (or substantially continuously) identify whether anyrelevant dataset is added or changed (beyond the creation of dataset 442a), and initiate a messaging service (e.g., via an activity feed) tonotify user 408 b of such events. According to some examples, atomizeddataset 442 a may be formed as triples compliant with an RDFspecification, and repository 440 may be a database storage deviceformed as a “triplestore.” While dataset 442 a, public dataset 442 b,and private dataset 442 c are described above as separately partitiongraphs that may be linked to form consolidated datasets and graphs(e.g., at query time, or during any other data operation), dataset 442 amay be integrated with either public dataset 442 b or private dataset442 c, or both, to form a physically contiguous data arrangement orgraph (e.g., a unitary graph without links), according to at least oneexample.

FIG. 5 is a diagram depicting a flow chart to perform an operation of acollaborative dataset consolidation system, according to someembodiments. Diagram 500 depicts a flow for an example of forming andquerying collaborative datasets in association with a collaborativedataset consolidation system. At 502, a data file including a datasetmay be received into a collaborative dataset consolidation system, andthe dataset may be formatted at 504 to form an atomized dataset (e.g., afirst atomized dataset). The atomized dataset may include atomized datapoints, whereby each atomized data point may include data representingat least two objects (e.g., a subject and an object of a “triple) and anassociation (e.g., a predicate) between the two objects. At 506, anotheratomized dataset (e.g., a second atomized dataset) may be formed toinclude the first atomized dataset and one or more other atomizeddatasets. For example, a consolidated dataset, as a second atomizeddataset, may include the atomized dataset linked to other atomizeddatasets. In some cases, other datasets, such as differently-formatteddatasets may be converted to a similar format so that the datasets mayinteroperate with each other as well as the data set of 504. Thus, anatomized dataset may be formed (e.g., as a consolidated dataset) bylinking one or more atomized datasets to the dataset of 504. Accordingto some embodiments, 506 and related functionalities may be optional. At508, data representing a query may be received into the collaborativedataset consolidation system. The query may be associated with anidentifier, which may be an attribute of a user, a dataset, or any othercomponent or element associated with a collaborative datasetconsolidated system. At 510, a subset of another atomized datasetrelevant to the query may be identified. Here, some portions of theother dataset may be disposed in different data repositories. Forexample, one or more portions of a second atomized dataset may beidentified as being relevant to a query or sub-query. Multiple relevantportions of the second atomized dataset may reside or may be stored indifferent databases or data stores. At 512, sub-queries may be generatedsuch that each may be configured to access at least one of the differentdata repositories. For example a first sub-query may be applied (e.g.,re-written) to access a first type of triplestore (e.g., a triplestorearchitected to function as a BLAZEGRAPH triplestore, which is developedby Systap, LLC of Washington, D.C., U.S.A.), a second sub-query may beconfigured to access a second type of triple store (e.g., a triplestorearchitected to function as a STARDOG triplestore, which is developed byComplexible, Inc. of Washington, D.C., U.S.A.), and a third sub-querymay be applied to access a first type of triplestore (e.g., atriplestore architected to function as a FUSEKI triplestore, which maybe maintained by The Apache Software Foundation of Forest Hill, Md.,U.S.A.). At 514, data representing query results from at least one ofthe different data repositories may be received. According to variousembodiments, the query may be re-written and applied to data storesserially (or substantially serially) or in parallel (or substantially inparallel), or in any combination thereof.

FIG. 6 is a diagram depicting an example of a dataset analyzer and aninference engine, according to some embodiments. Diagram 600 includes adataset ingestion controller 620, which, in turn, includes a datasetanalyzer 630 and a format converter 640. As shown, dataset ingestioncontroller 620 may be configured to receive data file 601 a, which mayinclude a dataset formatted in a specific format. An example of a formatincludes CSV, JSON, XML, XLS, XLS, MySQL, binary, RDF, or other similardata formats. Dataset analyzer 630 may be configured to analyze datafile 601 a to detect and resolve data entry exceptions (e.g., an imageembedded in a cell of a tabular file, missing annotations, etc.).Dataset analyzer 630 also may be configured to classify subsets of data(e.g., a column) in data file 601 a as a particular data type (e.g.,integers representing a year expressed in accordance with a Gregoriancalendar schema, five digits constitute a zip code, etc.), and the like.Dataset analyzer 630 can be configured to analyze data file 601 a tonote the exceptions in the processing pipeline, and to append, embed,associate, or link user interface features to one or more elements ofdata file 601 a to facilitate collaborative user interface functionality(e.g., at a presentation layer) with respect to a user interface.Further, dataset analyzer 630 may be configured to analyze data file 601a relative to dataset-related data to determine correlations amongdataset attributes of data file 601 a and other datasets 603 b (andattributes, such as metadata 603 a). Once a subset of correlations hasbeen determined, a dataset formatted in data file 601 a (e.g., as anannotated tabular data file, or as a CSV file) may be enriched, forexample, by associating links to the dataset of data file 601 a to formthe dataset of data file 601 b, which, in some cases, may have a similardata format as data file 601 a (e.g., with data enhancements,corrections, and/or enrichments). Note that while format converter 640may be configured to convert any CSV, JSON, XML, XLS, RDF, etc. intoRDF-related data formats, format converter 640 may also be configured toconvert RDF and non-RDF data formats into any of CSV, JSON, XML, XLS,MySQL, binary, XLS, RDF, etc. Note that the operations of datasetanalyzer 630 and format converter 640 may be configured to operate inany order serially as well as in parallel (or substantially inparallel). For example, dataset analyzer 630 may analyze datasets toclassify portions thereof, either prior to format conversion byformatter converter 640 or subsequent to the format conversion. In somecases, at least one portion of format conversion may occur duringdataset analysis performed by dataset analyzer 630.

Format converter 640 may be configured to convert dataset of data file601 b into an atomized dataset 601 c, which, in turn, may be stored insystem repositories 640 a that may include one or more atomized datastore (e.g., including at least one triplestore). Examples offunctionalities to perform such conversions may include, but are notlimited to, CSV2RDF data applications to convert CVS datasets to RDFdatasets (e.g., as developed by Rensselaer Polytechnic Institute andreferenced by the World Wide Web Consortium (“W3C”)), R2RML dataapplications (e.g., to perform RDB to RDF conversion, as maintained bythe World Wide Web Consortium (“W3C”)), and the like.

As shown, dataset analyzer 630 may include an inference engine 632,which, in turn, may include a data classifier 634 and a datasetenrichment manager 636. Inference engine 632 may be configured toanalyze data in data file 601 a to identify tentative anomalies and toinfer corrective actions, or to identify tentative data enrichments(e.g., by joining with other datasets) to extend the data beyond thatwhich is in data file 601 a. Inference engine 632 may receive data froma variety of sources to facilitate operation of inference engine 632 ininferring or interpreting a dataset attribute (e.g., as a derivedattribute) based on the analyzed data. Responsive to a request inputdata via data signal 601 d, for example, a user may enter a correctannotation into a user interface, which may transmit corrective data 601d as, for example, an annotation or column heading. Thus, the user maycorrect or otherwise provide for enhanced accuracy in atomized datasetgeneration “in-situ,” or during the dataset ingestion and/or graphformation processes. As another example, data from a number of sourcesmay include dataset metadata 603 (e.g., descriptive data or informationspecifying dataset attributes), dataset data 603 b (e.g., some or alldata stored in system repositories 640 a, which may store graph data),schema data 603 c (e.g., sources, such as schema.org, that may providevarious types and vocabularies), ontology data 603 d from any suitableontology (e.g., data compliant with Web Ontology Language (“OWL”), asmaintained by the World Wide Web Consortium (“W3C”)), and any othersuitable types of data sources.

In one example, data classifier 634 may be configured to analyze acolumn of data to infer a datatype of the data in the column. Forinstance, data classifier 634 may analyze the column data to infer thatthe columns include one of the following datatypes: an integer, astring, a time, etc., based on, for example, data from data 601 d, aswell as based on data from data 603 a to 603 d. In another example, dataclassifier 634 may be configured to analyze a column of data to infer adata classification of the data in the column (e.g., where inferring thedata classification may be more sophisticated than identifying orinferring a datatype). For example, consider that a column of ten (10)integer digits is associated with an unspecified or unidentifiedheading. Data classifier 634 may be configured to deduce the dataclassification by comparing the data to data from data 601 d, and fromdata 603 a to 603 d. Thus, the column of unknown 10-digit data in data601 a may be compared to 10-digit columns in other datasets that areassociated with an annotation of “phone number.” Thus, data classifier634 may deduce the unknown 10-digit data in data 601 a includes phonenumber data.

In yet another example, inference engine 632 may receive data (e.g.,datatype or data classification, or both) from an attribute correlator663. As shown, attribute correlator 663 may be configured to receivedata, including attribute data, from dataset ingestion controller 620,from data sources (e.g., UI-related/user inputted data 601 d, and data603 a to 603 d), from system repositories 640 a, from external publicrepository 640 b, from external private repository 640 c, from dominiondataset attribute data store 662, from dominion user account attributedata store 662, and from any other sources of data. In the exampleshown, dominion dataset attribute data store 662 may be configured tostore dataset attribute data for most, a predominant amount, or all ofdata over which collaborative dataset consolidation system has dominion,whereas dominion user account attribute data store 662 may be configuredto store user or user account attribute data for most, a predominantamount, or all of the data in its domain.

Attribute correlator 663 may be configured to analyze the data to detectpatterns that may resolve an issue. For example, attribute correlator663 may be configured to analyze the data, including datasets, to“learn” whether unknown 10-digit data is likely a “phone number” ratherthan another data classification. In this case, a probability may bedetermined that a phone number is a more reasonable conclusion based on,for example, regression analysis or similar analyses. Further, attributecorrelator 663 may be configured to detect patterns or classificationsamong datasets and other data through the use of Bayesian networks,clustering analysis, as well as other known machine learning techniquesor deep-learning techniques. Attribute correlator 663 also may beconfigured to generate enrichment data 607 b that may includeprobabilistic or predictive data specifying, for example, a dataclassification or a link to other datasets to enrich a dataset.According to some examples, attribute correlator 663 may further beconfigured to analyze data in dataset 601 a, and based on that analysis,attribute correlator 663 may be configured to recommend or implement oneor more added columns of data. To illustrate, consider that attributecorrelator 663 may be configured to derive a specific correlation basedon data 607 a that describe three (3) columns, whereby those threecolumns are sufficient to add a fourth (4^(th)) column as a derivedcolumn. In some cases, the data in the 4^(th) column may be derivedmathematically via one or more formulae. Therefore, additional data maybe used to form, for example, additional “triples” to enrich or augmentthe initial dataset.

In yet another example, inference engine 632 may receive data (e.g.,enrichment data 607 b) from a dataset attribute manager 661, whereenrichment data 607 b may include derived data or link-related data toform consolidated datasets. Consider that attribute correlator 663 candetect patterns in datasets in repositories 640 a to 640 c, among othersources of data, whereby the patterns identify or correlate to a subsetof relevant datasets that may be linked with the dataset in data 601 a.The linked datasets may form a consolidated dataset that is enrichedwith supplemental information from other datasets. In this case,attribute correlator 663 may pass the subset of relevant datasets asenrichment data 607 b to dataset enrichment manager 636, which, in turn,may be configured to establish the links for a dataset in 601 b. Asubset of relevant datasets may be identified as a supplemental subsetof supplemental enrichment data 607 b. Thus, converted dataset 601 c(i.e., an atomized dataset) may include links to establish collaborativedataset formed with consolidated datasets.

Dataset attribute manager 661 may be configured to receive correlatedattributes derived from attribute correlator 663. In some cases,correlated attributes may relate to correlated dataset attributes basedon data in data store 662 or based on data in data store 664, amongothers. Dataset attribute manager 661 also monitors changes in datasetand user account attributes in respective repositories 662 and 664. Whena particular change or update occurs, collaboration manager 660 may beconfigured to transmit collaborative data 605 to user interfaces ofsubsets of users that may be associated the attribute change (e.g.,users sharing a dataset may receive notification data that the datasethas been updated or queried).

Therefore, dataset enrichment manager 636, according to some examples,may be configured identify correlated datasets based on correlatedattributes as determined, for example, by attribute correlator 663. Thecorrelated attributes, as generated by attribute correlator 663, mayfacilitate the use of derived data or link-related data, as attributes,to form associate, combine, join, or merge datasets to form consolidateddatasets. A dataset 601 b may be generated by enriching a dataset 601 ausing dataset attributes to link to other datasets. For example, dataset601 a may be enriched with data extracted from (or linked to) otherdatasets identified by (or sharing similar) dataset attributes, such asdata representing a user account identifier, user characteristics,similarities to other datasets, one or more other user accountidentifiers that may be associated with a dataset, data-relatedactivities associated with a dataset (e.g., identity of a user accountidentifier associated with creating, modifying, querying, etc. aparticular dataset), as well as other attributes, such as a “usage” ortype of usage associated with a dataset. For instance, a virus-relateddataset (e.g., Zika dataset) may have an attribute describing a contextor usage of dataset, such as a usage to characterize susceptiblevictims, usage to identify a vaccine, usage to determine an evolutionaryhistory of a virus, etc. So, attribute correlator 663 may be configuredto correlate datasets via attributes to enrich a particular dataset.

According to some embodiments, one or more users or administrators of acollaborative dataset consolidation system may facilitate curation ofdatasets, as well as assisting in classifying and tagging data withrelevant datasets attributes to increase the value of the interconnecteddominion of collaborative datasets. According to various embodiments,attribute correlator 663 or any other computing device operating toperform statistical analysis or machine learning may be configured tofacilitate curation of datasets, as well as assisting in classifying andtagging data with relevant datasets attributes. In some cases, datasetingestion controller 620 may be configured to implement third-partyconnectors to, for example, provide connections through whichthird-party analytic software and platforms (e.g., R, SAS, Mathematica,etc.) may operate upon an atomized dataset in the dominion ofcollaborative datasets.

FIG. 7 is a diagram depicting operation of an example of an inferenceengine, according to some embodiments. Diagram 700 depicts an inferenceengine 780 including a data classifier 781 and a dataset enrichmentmanager 783, whereby inference engine 780 is shown to operate on data706 (e.g., one or more types of data described in FIG. 6), and furtheroperates on annotated tabular data representations of dataset 702,dataset 722, dataset 742, and dataset 762. Dataset 702 includes rows 710to 716 that relate each population number 704 to a city 702. Dataset 722includes rows 730 to 736 that relate each city 721 to both ageo-location described with a latitude coordinate (“lat”) 724 and alongitude coordinate (“long”) 726. Dataset 742 includes rows 750 to 756that relate each name 741 to a number 744, whereby column 744 omits anannotative description of the values within column 744. Dataset 762includes rows, such as row 770, that relate a pair of geo-coordinates(e.g., latitude coordinate (“lat”) 761 and a longitude coordinate(“long”) 764) to a time 766 at which a magnitude 768 occurred during anearthquake.

Inference engine 780 may be configured to detect a pattern in the dataof column 704 in dataset 702. For example, column 704 may be determinedto relate to cities in Illinois based on the cities shown (or based onadditional cities in column 704 that are not shown, such as Skokie,Cicero, etc.). Based on a determination by inference engine 780 thatcities 704 likely are within Illinois, then row 716 may be annotated toinclude annotative portion (“IL”) 790 (e.g., as derived supplementaldata) so that Springfield in row 716 can be uniquely identified as“Springfield, Ill.” rather than, for example, “Springfield, Nebr.” or“Springfield, Mass.” Further, inference engine 780 may correlate columns704 and 721 of datasets 702 and 722, respectively. As such, eachpopulation number in rows 710 to 716 may be correlated to correspondinglatitude 724 and longitude 726 coordinates in rows 730 to 734 of dataset722. Thus, dataset 702 may be enriched by including latitude 724 andlongitude 726 coordinates as a supplemental subset of data. In the eventthat dataset 762 (and latitude 724 and longitude 726 data) are formatteddifferently than dataset 702, then latitude 724 and longitude 726 datamay be converted to an atomized data format (e.g., compatible with RDF).Thereafter, a supplemental atomized dataset can be formed by linking orintegrating atomized latitude 724 and longitude 726 data with atomizedpopulation 704 data in an atomized version of dataset 702. Similarly,inference engine 780 may correlate columns 724 and 726 of dataset 722 tocolumns 761 and 764. As such, earthquake data in row 770 of dataset 762may be correlated to the city in row 734 (“Springfield, Ill.”) ofdataset 722 (or correlated to the city in row 716 of dataset 702 via thelinking between columns 704 and 721). The earthquake data may be derivedvia lat/long coordinate-to-earthquake correlations as supplemental datafor dataset 702. Thus, new links (or triples) may be formed tosupplement population data 704 with earthquake magnitude data 768.

Inference engine 780 also may be configured to detect a pattern in thedata of column 741 in dataset 742. For example, inference engine 780 mayidentify data in rows 750 to 756 as “names” without an indication of thedata classification for column 744. Inference engine 780 can analyzeother datasets to determine or learn patterns associated with data, forexample, in column 741. In this example, inference engine 780 maydetermine that names 741 relate to the names of “baseball players.”Therefore, inference engine 780 determines (e.g., predicts or deduces)that numbers in column 744 may describe “batting averages.” As such, acorrection request 796 may be transmitted to a user interface to requestcorrective information or to confirm that column 744 does includebatting averages. Correction data 798 may include an annotation (e.g.,batting averages) to insert as annotation 794, or may include anacknowledgment to confirm “batting averages” in correction request data796 is valid. Note that the functionality of inference engine 780 is notlimited to the examples describe in FIG. 7 and is more expansive than asdescribed in the number of examples.

FIG. 8 is a diagram depicting a flow chart as an example of ingesting anenhanced dataset into a collaborative dataset consolidation system,according to some embodiments. Diagram 800 depicts a flow for an exampleof inferring dataset attributes and generating an atomized dataset in acollaborative dataset consolidation system. At 802, data representing adataset having a data format may be received into a collaborativedataset consolidation system. The dataset may be associated with anidentifier or other dataset attributes with which to correlate thedataset. At 804, a subset of data of the dataset is interpreted againstsubsets of data (e.g., columns of data) for one or more dataclassifications (e.g., datatypes) to infer or derive at least aninferred attribute for a subset of data (e.g., a column of data). Insome examples, the subset of data may relate to a columnarrepresentation of data in an annotated tabular data format, or CSV file.At 806, the subset of the data may be associated with annotative dataidentifying the inferred attribute. Examples of an inferred attributeinclude the inferred “baseball player” names annotation and the inferred“batting averages” annotation, as described in FIG. 7. At 808, thedataset is converted from the data format to an atomized dataset havinga specific format, such as an RDF-related data format. The atomizeddataset may include a set of atomized data points, whereby each datapoint may represented as a RDF triple. According to some embodiments,inferred dataset attributes may be used to identify subsets of data inother dataset, which may be used to extend or enrich a dataset. Anenriched dataset may be stored as data representing “an enriched graph”in, for example, a triplestore or an RDF store (e.g., based on agraph-based RDF model). In other cases, enriched graphs formed inaccordance with the above may be stored in any type of data store orwith any database management system.

FIG. 9 is a diagram depicting another example of a dataset ingestioncontroller, according to various embodiments. Diagram 900 depicts adataset ingestion controller 920 including a dataset analyzer 930, adata storage manager 938, a format converter 940, and an atomizeddata-based workflow loader 945. Further, dataset ingestion controller920 is configured to load atomized data points in an atomized dataset901 c into an atomized data point store 950, which, in some examples,may be implemented as a triplestore. According to some examples,elements depicted in diagram 900 of FIG. 9 may include structures and/orfunctions as similarly-named or similarly-numbered elements depicted inother drawings.

Data storage manager 938 may be configured to build a corpus ofcollaborative datasets by, for example, forming “normalized” data filesin a collaborative dataset consolidation system, such that a normalizeddata file may be represented as follows:

/hash/XXX,

-   -   where “hash” may be a hashed representation as a filename (i.e.,        a reduced or compressed representation of the data), whereby a        filename may be based on, for example, a hash value of the bites        in the raw data, and    -   where XXX indicates either “raw” (e.g., raw data), “treatment*”        (e.g., a treatment file that specifies treatments applied to        data, such as identifying each column, etc.) or “meta*” (e.g.,        an amount of metadata).        Further, data storage manager 938 may configure dataset versions        to hold an original file name as a pointer to a storage        location. In accordance with some examples, identical original        files need be stored one time in atomized data point store 950.        Data storage manager 938 may operate to normalize data files        into a graph of triples, whereby each dataset version may be        loaded into a graph database instance. Also, data storage        manager 938 may be configured to maintain searchable endpoints        for dataset 910 over one or more versions (e.g.,        simultaneously).

An example of a data model with which data storage manager 938 storesdata is shown as data model 909. In this model, a dataset 910 may betreated as versions (V0) 912, (V1) 912 b and (Vn) 912 n, and versionsmay be treated as records or files (f0) 911, (f1) 913, (f2) 915, (f3)917, and (f4) 919. Dataset 910 may include a directed graph of datasetversions and a set of named references to versions within the dataset. Adataset version 912 may contain a hierarchy of named files, each with aname unique within a version and a version identifier. The datasetversion may reference a data file (e.g., 911 to 919). A data filerecord, or file, they referred to an “original” data file (e.g., the rawuser-provided bytes), and any “treatments” to the file that are storedalongside original files these treatments can include, for example aconverted file containing the same data represented as triples, or aschema or metadata about the file. In the example shown for data model909, version 912 a may include a copy of a file 911. A next version 912b is shown to include copies of files 913 and 915, as well as includinga pointer 918 to file 911, whereas a subsequent version 912 n is shownto include copies of files 917 and 919, as well as pointers 918 to files911, 913, and 915.

Version controller 939 may be configured to manage the versioning ofdataset 910 by tracking each version as an “immutable” collection ofdata files and pointers to data files. As the dataset versions areconfigured to be immutable, when dataset 910 is modified, versioncontroller 939 provides for a next version, whereby new data (e.g.,changed data) is stored in a file and pointers to previous files areidentified.

Atomized data-based workflow loader 945, according to some examples, maybe configured to load graph data onto atomized data point store 950(e.g., a triplestore) from disk (e.g., an S3 Amazon® cloud storageserver).

FIG. 10 is a diagram depicting a flow chart as an example of managingversioning of dataset, according to some embodiments. Diagram 1000depicts a flow for generating, for example, an immutable next version ina collaborative dataset consolidation system. At 1002, data representinga dataset (e.g., a first dataset) having a data format may be receivedinto a collaborative dataset consolidation system. At 1004, datarepresenting attributes associated with the dataset may also bereceived. The attributes may include an account identifier or otherdataset or user account attributes. At 1006, a first version of thedataset associated with a first subset of atomized data points isidentified. In some cases, the first subset of atomized data points maybe stored in a graph or any other type of database (e.g., atriplestore). A subset of data that varies from the first version of thedataset is identified at 1008. In some examples, the subset of data thatvaries from the first version may be modified data of the first dataset,or the subset of data may be data from another dataset that isintegrated or linked to the first dataset. In some cases, the subset ofdata that varies from the first version is being added or deleted fromthat version to form another version. At 1010, the subset of data may beconverted to a second subset of atomized data points, which may have aspecific format similar to the first subset. The subset of data may beanother dataset that is converted into the specific format. For example,both may be in triples format.

At 1012, a second version of the dataset is generated to include thefirst subset of atomized data points and the second subsets of atomizeddata points. According to some examples, the first version and secondversion persist as immutable datasets that may be referenced at any ormost times (e.g., a first version may be cited as being relied on in aquery that contributes to published research results regardless of asecond or subsequent version). Further, a second version need notinclude a copy of the first subset of atomized data points, but rathermay store a pointer the first subset of atomized data points along withthe second subsets of atomized data points. Therefore, subsequentversion may be retained without commensurate increases in memory tostore subsequent immutable versions, according to some embodiments.Note, too, that the second version may include the second subsets ofatomized data points as a protected dataset that may be authorized forinclusion into the second version (i.e., a user creating the secondversion may need authorization to include the second subsets of atomizeddata points). At 1014, the first subset of atomized data points and thesecond subset of atomized data points as an atomized dataset are storedin one or more repositories. Therefore, multiple sources of data mayprovide differently-formatted datasets, whereby flow 1000 may beimplemented to transform the formats of each dataset to facilitateinteroperability among the transformed datasets. According to variousexamples, more or fewer of the functionalities set forth in flow 1000may be omitted or maybe enhanced.

FIG. 11 is a diagram depicting an example of an atomized data-basedworkflow loader, according to various embodiments. Diagram 1100 depictsan atomized data-based workflow loader 1145 that is configured todetermine which type of database or data store (e.g., triplestore) for aparticular dataset that is be loaded. As shown, workflow loader 1145includes a dataset requirement determinator 1146 and a product selector1148. Dataset requirement determinator 1146 may be configured todetermine the loading and/or query requirements for a particulardataset. For example, a particular dataset may include time-series data,GPU-related processing data, geo-spatial-related data, etc., any ofwhich may be implemented optimally on data store 1150 (e.g., data store1150 has certain product features that are well-suited for processingthe particular dataset), but may be suboptimally implemented on datastore 1152. Once the requirements are determined by dataset requirementdeterminator 1146, product selector 1148 is configured to select aproduct, such as triple store (type 1) 1150 for loading the dataset.Next, product selector 1148 can transmit the dataset 1101 a for loadinginto product 1150. Examples of one or more of triplestores 1150 to 1152may include one or more of a BLAZEGRAPH triplestore, a STARDOGtriplestore, or a FUSEKI triplestore, all of which have been describedabove. Therefore, workflow loader 1145 may be configured to selectBLAZEGRAPH triplestore, a STARDOG triplestore, or a FUSEKI triplestorebased on each database's capabilities to perform queries in particulartypes of data and datasets.

Data model 1190 includes a data package representation 1110 that may beassociated with a source 1112 (e.g., a dataset to be loaded) and aresource 1111 (e.g., data representations of a triplestore). Thus, datarepresentation 1160 may model operability of “how to load” datasets intoa graph 114, whereas data representation 1162 may model operability of“what to load.” As shown, data representation 1162 may include aninstance 1120, one or more references to a data store 1122, and one ormore references to a product 1124. In at least one example, datarepresentation 1162 may be equivalent to dataset requirementdeterminator 1146, whereas data representation 1160 may be equivalent toproduct selector 1148.

FIG. 12 is a diagram depicting a flow chart as an example of loading anatomized dataset into an atomized data point store, according to someembodiments. Flow 1200 may begin at 1202, at which an atomized dataset(e.g., a triple) is received in preparation to load into a data store(e.g., a triplestore). At 1204, resource requirements data is determinedto describe at least one resource requirement. For example, a resourcerequirement may describe one or more necessary abilities of atriplestore to optimal load and provide graph data. In at least onecase, a dataset being loaded by a loader may be optimally used onparticular type of data store (e.g., a triplestore configured optimallyhandle text searches, geo-spatial information, etc.). At 1206, aparticular data store is selected based on an ability or capability ofthe particular data store to fulfill a requirement to operate anatomized data point store (or triplestore). At 1208, a load operation ofthe atomized dataset is performed into the data store.

FIG. 13 is a diagram depicting an example of a dataset query engine,according to some embodiments. Diagram 1300 shows a dataset query engine1330 disposed in a collaborative dataset consolidation system 1310.According to some examples, elements depicted in diagram 1300 of FIG. 13may include structures and/or functions as similarly-named orsimilarly-numbered elements depicted in other drawings. Dataset queryengine 1330 may receive a query to apply to any number of atomizeddatasets in one or more repositories, such as data stores 1350, 1351,and 1352, within or without collaborative dataset consolidation system1310. Repositories may include those that include linked externaldatasets (e.g., including imported external datasets, such if protecteddatasets are imported, whereby restrictions may remain (e.g., securitylogins)). In some cases, there may be an absence of standards with whichto load and manage atomized datasets that may be loaded into disparatedata stores. According to some examples, dataset query engine 1330 maybe configured to propagate queries, such as queries 1301 a, 1301 b, and1301 c as a federated query 1360 of different datasets disposed overdifferent data schema. Therefore, dataset query engine 1330 may beconfigured to propagate federated query 1360 over differenttriplestores, each of which may be architected to have differentcapabilities and functionalities to implement a triplestore.

According to one example, dataset query engine 1330 may be configured toanalyze the query to classify portions to form classified query portions(e.g., portions of the query that are classified against categorizationschema). Dataset query engine 1330 may be configured to re-write (e.g.,partition) the query into a number of query portions based on, forexample, the classification type of each query portion. Thus, datasetquery engine 1330 may receive a query result from distributed datarepositories, at least a portion of which may include disparatedistributed triplestores.

In some cases, the query may originate as a user query 1302. That is, auser associated with the user account identifier may submit via acomputing device user query 1302. In this case, user query 1302 may havebeen authenticated to access collaborative data consolidation system1330 generally, or to the extent in which permissions and privilegeshave been granted as defined by, for example, data representing a useraccount. In other cases, the query may originate as anexternally-originated query 1303. Here, an external computing devicehosting an external dataset that is linked to an internal dataset (e.g.,a dataset disposed in an internal data store 1350) may apply its queryto data secretary engine 1330 (e.g., without user account-levelauthentication that typically is applied to user queries 1302). Notethat dataset query engine 1330 may be configured to perform query-levelauthorization processes to ensure authorization of user queries 1302 andexternally-originated queries 1303.

Further to diagram 1300, dataset query engine 1330 is shown to include aparser 1332, a validator 1334, a query classifier 1336, a sub-querygenerator 1338, and a query director 1339. According to some examples,parser 1332 may be configured to parse queries (e.g., queries 1302 and1303) to, among other things, identify one or more datasets subject tothe query. Validator 1334 may be configured to receive data representingthe identification of each of the datasets subject to the query, and maybe further configured to provide per-dataset authorization. For example,the level of authorization for applying queries 1302 and 1303 may bedetermined by analyzing each dataset against credentials or otherauthenticating data associated with a computing device or user applyingthe query. In one instance, if any authorization to access at least onedataset of any number of datasets (related to the query) may besufficient to reject query.

Query classifier 1336 may be configured to analyze each of theidentified datasets to classify each of the query portions directed tothose datasets. Thus, a number of query portions may be classified thesame or differently in accordance with a classification type. Accordingto one classification type, query classifier 1336 may be configured todetermine a type of repository (e.g., a type of data store, such as“type 1,” “type 2,” and “type n,”) associated with a portion of a query,and classify a query portion to be applied the particular type ofrepository. In at least one example, the different types of repositorymay include different triplestores, such as a BLAZEGRAPH triplestore, aSTARDOG triplestore, a FUSEKI triplestore, etc. Each type may indicatethat each database may have differing capabilities or approaches toperform queries in a particular manner.

According to another classification type, query classifier 1336 may beconfigured to determine a type of query associated with a query portion.For example, a query portion may related to transactional queries,analytic queries regarding geo-spatial data, queries related totime-series data, queries related to text searches, queries related tographic processing unit (“GPU”)-optimized data, etc. In some cases, suchtypes of data are loaded into specific types of repositories that areoptimally-suited to provide queries of specific types of data.Therefore, query classifier 1336 may classify query portions relative tothe types of datasets and data against which the query is applied.According to yet another classification type, query classifier 1336 maybe configured to determine a type of query associated with a queryportion to an external dataset. For example, a query portion may beidentified as being applied to an external dataset. Thus, a queryportion may be configured accordingly for application to them externaldatabase. Other classification query classification types are within thescope of the various embodiments and examples. In some cases, queryclassifier 1336 may be configured to classify a query with still yetanother type of query based on whether a dataset subject to a query isassociated with a specific entity (e.g., a user that owns the dataset,or an authorized user), or whether the dataset to be queried is securedsuch that a password or other authorization credentials may be required.

Sub-query generator 1338 may be configured to generate sub-queries thatmay be applied as queries 1301 a to 130 c, as directed by query director1339. In some examples, sub-query generate 1338 may be configured tore-write queries 1302 and 1303 to apply portions of the queries tospecific data stores 1350 to 1352 to optimize querying of data secretaryengine 1330. According to some examples, query director 1339, or anycomponent of dataset query engine 1330 (and including dataset queryengine 1330), may be configured to implement SPARQL as maintained by theW3C Consortium, or any other compliant variant thereof. In someexamples, dataset query engine 1330 may not be limited to theaforementioned and may implement any suitable query language. In someexamples, dataset query engine 1330 or portions thereof may beimplemented as a “query proxy” server or the like.

FIG. 14 is a diagram depicting a flow chart as an example of querying anatomized dataset stored in an atomized data point store, according tosome embodiments. Flow 1400 may begin at 1402, at which datarepresenting a query of a consolidated dataset is received into acollaborative dataset consolidation system, the consolidated datasetbeing stored in an atomized data store. The query may apply to a numberof datasets formatted as atomized datasets that are stored in one ormore atomized data stores (e.g., one or more triplestores). At 1404, thequery is analyzed to classify portions of the query to form classifiedquery portions. At 1406, the query may be partitioned (e.g., rewritten)into a number of queries or sub-queries as a function of aclassification type. For example, each of the sub-queries may berewritten or partitioned based on each of the classified query portions.For example, a sub-query may be re-written for transmission to arepository based on a type of repository describing the repository(e.g., one of any type of data store or database technologies, includingone of any type of triplestore). At 1408, data representing a queryresult may be retrieved from distributed data repositories. In someexamples, the query is a federated query of atomized data stores. Afederated query may represent multiple queries (e.g., in parallel, orsubstantially in parallel), according to some examples. In one instance,a federated query may be a SPARQL query executed over a federated graph(e.g., a family of RDF graphs).

FIG. 15 is a diagram depicting an example of a collaboration managerconfigured to present collaborative information regarding collaborativedatasets, according to some embodiments. Diagram 1500 depicts acollaboration manager 960 including a dataset attribute manager 961, andcoupled to a collaborative activity repository 1536. In this example,dataset attribute manager 961 is configured to monitor updates andchanges to various subsets of data representing dataset attribute data1534 a and various subsets of data representing user attribute data 1534b, and to identify such updates and changes. Further, dataset attributemanager 961 can be configured to determine which users, such as user1508, ought to be presented with activity data for presentation via acomputing device 1509 in a user interface 1518. In some examples,dataset attribute manager 961 can be configured to manage datasetattributes associated with one or more atomized datasets. For example,dataset attribute manager 961 can be configured to analyzing atomizeddatasets and, for instance, identify a number of queries associated witha atomized dataset, or a subset of account identifiers (e.g., of otherusers) that include descriptive data that may be correlated to theatomized dataset. To illustrate, consider that other users associatedwith other account identifiers have generated their own datasets (andmetadata), whereby the metadata may include descriptive data (e.g.,attribute data) that may be used to generate notifications to interestedusers of changes or modifications or activities related to a particulardataset. The notifications may be generated as part of an activity feedpresented in a user interface, in some examples.

Collaboration manager 960 receives the information to be presented to auser 1508 and causes it to be presented at computing device 1509. As anexample, the information presented may include a recommendation to auser to review a particular dataset based on, for example, similaritiesin dataset attribute data (e.g., users interested in Zika-based datasetsgenerated in Brazil may receive recommendation to access a dataset withthe latest dataset for Zika cases in Sao Paulo, Brazil). Note the listedtypes of attribute data monitored by dataset attribute manager 961 arenot intended to be limiting. Therefore, collaborative activityrepository 1536 may store other attribute types and attribute-relatedthan is shown.

FIG. 16 illustrates examples of various computing platforms configuredto provide various functionalities to components of a collaborativedataset consolidation system, according to various embodiments. In someexamples, computing platform 1600 may be used to implement computerprograms, applications, methods, processes, algorithms, or othersoftware, as well as any hardware implementation thereof, to perform theabove-described techniques.

In some cases, computing platform 1600 or any portion (e.g., anystructural or functional portion) can be disposed in any device, such asa computing device 1690 a, mobile computing device 1690 b, and/or aprocessing circuit in association with forming and queryingcollaborative datasets generated and interrelated according to variousexamples described herein.

Computing platform 1600 includes a bus 1602 or other communicationmechanism for communicating information, which interconnects subsystemsand devices, such as processor 1604, system memory 1606 (e.g., RAM,etc.), storage device 1608 (e.g., ROM, etc.), an in-memory cache (whichmay be implemented in RAM 1606 or other portions of computing platform1600), a communication interface 1613 (e.g., an Ethernet or wirelesscontroller, a Bluetooth controller, NFC logic, etc.) to facilitatecommunications via a port on communication link 1621 to communicate, forexample, with a computing device, including mobile computing and/orcommunication devices with processors, including database devices (e.g.,storage devices configured to store atomized datasets, including, butnot limited to triplestores, etc.). Processor 1604 can be implemented asone or more graphics processing units (“GPUs”), as one or more centralprocessing units (“CPUs”), such as those manufactured by Intel®Corporation, or as one or more virtual processors, as well as anycombination of CPUs and virtual processors. Computing platform 1600exchanges data representing inputs and outputs via input-and-outputdevices 1601, including, but not limited to, keyboards, mice, audioinputs (e.g., speech-to-text driven devices), user interfaces, displays,monitors, cursors, touch-sensitive displays, LCD or LED displays, andother I/O-related devices.

Note that in some examples, input-and-output devices 1601 may beimplemented as, or otherwise substituted with, a user interface in acomputing device associated with a user account identifier in accordancewith the various examples described herein.

According to some examples, computing platform 1600 performs specificoperations by processor 1604 executing one or more sequences of one ormore instructions stored in system memory 1606, and computing platform1600 can be implemented in a client-server arrangement, peer-to-peerarrangement, or as any mobile computing device, including smart phonesand the like. Such instructions or data may be read into system memory1606 from another computer readable medium, such as storage device 1608.In some examples, hard-wired circuitry may be used in place of or incombination with software instructions for implementation. Instructionsmay be embedded in software or firmware. The term “computer readablemedium” refers to any tangible medium that participates in providinginstructions to processor 1604 for execution. Such a medium may takemany forms, including but not limited to, non-volatile media andvolatile media. Non-volatile media includes, for example, optical ormagnetic disks and the like. Volatile media includes dynamic memory,such as system memory 1606.

Known forms of computer readable media includes, for example, floppydisk, flexible disk, hard disk, magnetic tape, any other magneticmedium, CD-ROM, any other optical medium, punch cards, paper tape, anyother physical medium with patterns of holes, RAM, PROM, EPROM,FLASH-EPROM, any other memory chip or cartridge, or any other mediumfrom which a computer can access data. Instructions may further betransmitted or received using a transmission medium. The term“transmission medium” may include any tangible or intangible medium thatis capable of storing, encoding or carrying instructions for executionby the machine, and includes digital or analog communications signals orother intangible medium to facilitate communication of suchinstructions. Transmission media includes coaxial cables, copper wire,and fiber optics, including wires that comprise bus 1602 fortransmitting a computer data signal.

In some examples, execution of the sequences of instructions may beperformed by computing platform 1600. According to some examples,computing platform 1600 can be coupled by communication link 1621 (e.g.,a wired network, such as LAN, PSTN, or any wireless network, includingWiFi of various standards and protocols, Bluetooth®, NFC, Zig-Bee, etc.)to any other processor to perform the sequence of instructions incoordination with (or asynchronous to) one another. Computing platform1600 may transmit and receive messages, data, and instructions,including program code (e.g., application code) through communicationlink 1621 and communication interface 1613. Received program code may beexecuted by processor 1604 as it is received, and/or stored in memory1606 or other non-volatile storage for later execution.

In the example shown, system memory 1606 can include various modulesthat include executable instructions to implement functionalitiesdescribed herein. System memory 1606 may include an operating system(“O/S”) 1632, as well as an application 1636 and/or logic module(s)1659. In the example shown in FIG. 16, system memory 1606 may include adataset ingestion controller modules 1652 and/or its components (e.g., adataset analyzer module 1752, an inference engine module 1754, and aformat converter module 1756), any of which, or one or more portions ofwhich, can be configured to facilitate any one or more components of acollaborative dataset consolidation system by implementing one or morefunctions described herein. Further, system memory 1606 may include adataset query engine module 1654 and/or its components (e.g., a parsermodule 1852, a validator module 1854, a sub-query generator module 1856,and the query classifier module 1858), any of which, or one or moreportions of which, can be configured to facilitate any one or morecomponents of a collaborative dataset consolidation system byimplementing one or more functions described herein. Additionally,system memory 1606 may include a collaboration manager module 1656and/or any of its components that can be configured to facilitate anyone or more components of a collaborative dataset consolidation systemby implementing one or more functions described herein.

The structures and/or functions of any of the above-described featurescan be implemented in software, hardware, firmware, circuitry, or acombination thereof. Note that the structures and constituent elementsabove, as well as their functionality, may be aggregated with one ormore other structures or elements. Alternatively, the elements and theirfunctionality may be subdivided into constituent sub-elements, if any.As software, the above-described techniques may be implemented usingvarious types of programming or formatting languages, frameworks,syntax, applications, protocols, objects, or techniques. As hardwareand/or firmware, the above-described techniques may be implemented usingvarious types of programming or integrated circuit design languages,including hardware description languages, such as any register transferlanguage (“RTL”) configured to design field-programmable gate arrays(“FPGAs”), application-specific integrated circuits (“ASICs”), or anyother type of integrated circuit. According to some embodiments, theterm “module” can refer, for example, to an algorithm or a portionthereof, and/or logic implemented in either hardware circuitry orsoftware, or a combination thereof. These can be varied and are notlimited to the examples or descriptions provided.

In some embodiments, modules 1652, 1654, and 1656 of FIG. 16, or one ormore of their components, or any process or device described herein, canbe in communication (e.g., wired or wirelessly) with a mobile device,such as a mobile phone or computing device, or can be disposed therein.

In some cases, a mobile device, or any networked computing device (notshown) in communication with one or more modules 1659 (modules 1652,1654, and 1656 of FIG. 16) or one or more of its/their components (orany process or device described herein), can provide at least some ofthe structures and/or functions of any of the features described herein.As depicted in the above-described figures, the structures and/orfunctions of any of the above-described features can be implemented insoftware, hardware, firmware, circuitry, or any combination thereof.Note that the structures and constituent elements above, as well astheir functionality, may be aggregated or combined with one or moreother structures or elements. Alternatively, the elements and theirfunctionality may be subdivided into constituent sub-elements, if any.As software, at least some of the above-described techniques may beimplemented using various types of programming or formatting languages,frameworks, syntax, applications, protocols, objects, or techniques. Forexample, at least one of the elements depicted in any of the figures canrepresent one or more algorithms. Or, at least one of the elements canrepresent a portion of logic including a portion of hardware configuredto provide constituent structures and/or functionalities.

For example, modules 1652, 1654, and 1656 of FIG. 16 or one or more ofits/their components, or any process or device described herein, can beimplemented in one or more computing devices (i.e., any mobile computingdevice, such as a wearable device, such as a hat or headband, or mobilephone, whether worn or carried) that include one or more processorsconfigured to execute one or more algorithms in memory. Thus, at leastsome of the elements in the above-described figures can represent one ormore algorithms. Or, at least one of the elements can represent aportion of logic including a portion of hardware configured to provideconstituent structures and/or functionalities. These can be varied andare not limited to the examples or descriptions provided.

As hardware and/or firmware, the above-described structures andtechniques can be implemented using various types of programming orintegrated circuit design languages, including hardware descriptionlanguages, such as any register transfer language (“RTL”) configured todesign field-programmable gate arrays (“FPGAs”), application-specificintegrated circuits (“ASICs”), multi-chip modules, or any other type ofintegrated circuit.

For example, modules 1652, 1654, and 1656 of FIG. 16, or one or more ofits/their components, or any process or device described herein, can beimplemented in one or more computing devices that include one or morecircuits. Thus, at least one of the elements in the above-describedfigures can represent one or more components of hardware. Or, at leastone of the elements can represent a portion of logic including a portionof a circuit configured to provide constituent structures and/orfunctionalities.

According to some embodiments, the term “circuit” can refer, forexample, to any system including a number of components through whichcurrent flows to perform one or more functions, the components includingdiscrete and complex components. Examples of discrete components includetransistors, resistors, capacitors, inductors, diodes, and the like, andexamples of complex components include memory, processors, analogcircuits, digital circuits, and the like, including field-programmablegate arrays (“FPGAs”), application-specific integrated circuits(“ASICs”). Therefore, a circuit can include a system of electroniccomponents and logic components (e.g., logic configured to executeinstructions, such that a group of executable instructions of analgorithm, for example, and, thus, is a component of a circuit).According to some embodiments, the term “module” can refer, for example,to an algorithm or a portion thereof, and/or logic implemented in eitherhardware circuitry or software, or a combination thereof (i.e., a modulecan be implemented as a circuit). In some embodiments, algorithms and/orthe memory in which the algorithms are stored are “components” of acircuit. Thus, the term “circuit” can also refer, for example, to asystem of components, including algorithms. These can be varied and arenot limited to the examples or descriptions provided.

Although the foregoing examples have been described in some detail forpurposes of clarity of understanding, the above-described inventivetechniques are not limited to the details provided. There are manyalternative ways of implementing the above-described inventiontechniques. The disclosed examples are illustrative and not restrictive.

The invention claimed is:
 1. A method comprising: receiving, into acollaborative dataset consolidation system, data representing a query ofa consolidated dataset comprising a plurality of datasets formatted asplurality of atomized datasets; analyzing the query to classify portionsof the query to form classified query portions; partitioning the queryinto a plurality of sub-queries as a function of a classification typeassociated with each of the classified query portions, at least oneclassified query portion being classified by a type of dataset;determining a type of triple store loaded based on the type of dataset;executing the query and the sub-queries as a federated query to one ormore of the plurality of datasets using one or more triple stores storedin one or more distributed data repositories configured to be stored andaccessed by the collaborative data consolidation system, each of the oneor more triple stores being further formatted to query using a formatassociated with at least one of the one or more of the plurality of datasets, at least one of the one or more triple stores being the type oftriple store; retrieving responsive data representing a query resultreturned in response to the query or sub-queries from at least one ofthe one or more distributed data repositories; identifying the queryresults generated by the query as a data-related activity associatedwith a subset of datasets in the plurality of datasets; identifying oneor more user account identifiers associated with the subsets of datasetsassociated with the query; and disseminating an update as thedata-related activity to a community of networked users associated withthe one or more user account identifiers.
 2. The method of claim 1wherein disseminating the update to the community of networked userscomprises: generating data representing a notification to a userassociated with the one or more other account identifiers.
 3. The methodof claim 2 wherein generating data representing the notificationcomprises: causing presentation of the notification in an activity feedportion of a user interface of a computing device.
 4. The method ofclaim 1 wherein analyzing the query comprises: detecting discovery ofmodified data associated with the data-related activity.
 5. The methodof claim 4 wherein detecting discovery of the modified data associatedwith the data-related activity comprises: determining an activity ofquerying the dataset.
 6. The method of claim 4 wherein detectingdiscovery of the modified data associated with the data-related activitycomprises: determining an activity of modifying the dataset.
 7. Themethod of claim 4 further comprising: generating data representing arecommendation.
 8. The method of claim 1 further comprising: identifyinga number of atomized datasets associated with the query.
 9. The methodof claim 8 further comprising: implementing per-dataset permissions todetermine whether a user account is authorized to query each of theatomized datasets.
 10. The method of claim 8 further comprising:identifying an identifier associated with the query; determiningauthorization data between the identifier and each of atomized datasetsin the number of atomized datasets; and granting authorization to applythe query to the number of atomized datasets based on authorization toeach of the atomized datasets.
 11. A collaborative dataset consolidationsystem, comprising: a data store configured to receive, into thecollaborative dataset consolidation system, data representing a query ofa consolidated dataset comprising a plurality of datasets formatted asplurality of atomized datasets; and a processor configured to executeinstructions to implement a dataset query engine configured to: analyzethe query to classify portions of the query to form classified queryportions; partition the query into a plurality of sub-queries as afunction of a classification type associated with each of the classifiedquery portions, at least one classified query portion being classifiedby a type of dataset; determine a type of triple store loaded based onthe type of dataset; execute the query and the sub-queries as afederated query to one or more of the plurality of datasets using one ormore triple stores stored in one or more distributed data repositoriesconfigured to be stored and accessed by the collaborative dataconsolidation system, each of the one or more triple stores beingfurther formatted to query using a format associated with at least oneof the one or more of the plurality of data sets, at least one of theone or more triple stores being the type of triple store; retrieveresponsive data representing a query result returned in response to thequery or sub-queries from at least one of the one or more distributeddata repositories, identify the query results generated by the query asa data-related activity associated with a subset of datasets in theplurality of datasets; identify one or more user account identifiersassociated with the subsets of datasets associated with the query; anddisseminate an update as the data-related activity to a community ofnetworked users associated with the one or more user accountidentifiers.
 12. The collaborative dataset consolidation system of claim11, wherein the processor configured to disseminate the update to thecommunity of networked users is further configured to: generate datarepresenting a notification to a user associated with the one or moreother account identifiers.
 13. The collaborative dataset consolidationsystem of claim 12, wherein the processor configured to generate datarepresenting the notification is further configured to: causepresentation of the notification in an activity feed portion of a userinterface of a computing device.
 14. The collaborative datasetconsolidation system of claim 11, wherein the processor configured toanalyze the query is further configured to: detect discovery of modifieddata associated with the data-related activity.
 15. The collaborativedataset consolidation system of claim 14, wherein the processorconfigured to detect discovery of the modified data associated with thedata-related activity is further configured to: determine an activity ofquerying the dataset.
 16. The collaborative dataset consolidation systemof claim 14, wherein the processor configured to detect discovery of themodified data associated with the data-related activity is furtherconfigured to: determine an activity of modifying the dataset.
 17. Thecollaborative dataset consolidation system of claim 14, wherein theprocessor is further configured to: generate data representing arecommendation.
 18. The collaborative dataset consolidation system ofclaim 11, wherein the processor is further configured to: identify anumber of atomized datasets associated with the query.
 19. Thecollaborative dataset consolidation system of claim 18, wherein theprocessor is further configured to: implement per-dataset permissions todetermine whether a user account is authorized to query each of theatomized datasets.
 20. The collaborative dataset consolidation system ofclaim 18, wherein the processor is further configured to: identify anidentifier associated with the query; determine authorization databetween the identifier and each of atomized datasets in the number ofatomized datasets; and grant authorization to apply the query to thenumber of atomized datasets based on authorization to each of theatomized datasets.